Various systems have been investigated for further safety improvements in recent automotive industry. In particular, image sensors equipped with a camera are employed. The image sensors are used for identification of an object and motion analysis of the object through the image processing of the captured images.
Motion analysis may be performed using, for example, a correlation technique. The correlation technique is based on finding, in the two images corresponding to each other, a point (corresponding point) in one of the images that corresponds to a target point in another image. More specifically, according to this correlation technique, a template is selected in the preceding image, i.e., an image of interest out of, for example, two time-series images such that a target point is included in this template, and two or more windows of the same size as the image of interest are defined on the subsequent image, i.e., a reference image. Then, a correlation value (similarity) is computed between the template in the image of interest and each window in the reference image to find such a window that has the largest correlation value in the reference image. The center of gravity of that window is determined as the corresponding point. With more than two time-series images, after the corresponding point is found, this corresponding point is used as a new target point to find another corresponding point in the later-captured images. Repeated cycles of this allow tracking of trajectory of the corresponding points from frame to frame, and motion vectors can be computed in a sequential manner. A motion vector is a vector that connects a certain point in one frame to the corresponding point in another frame. A known example of the correlation technique is, for example, a SAD (Sum of Absolute Difference) algorithm.
Various methods have been proposed to compute motion vectors. For example, Patent Document 1 describes a process of detecting a corresponding point by using block matching to compute motion vectors. With this method, a target point is selected at a pixel level (on a pixel by pixel basis). The corresponding point is found at the pixel level as well. However, if similarity during the block matching operation is low, the corresponding point is searched at a sub-pixel level, such as ½ pixel and ¼ pixel, that is smaller than 1 pixel to compute motion vectors. This allows the corresponding point search with high accuracy at the sub-pixel level, and based on which the motion vectors are computed. Motion analysis can thus be performed with high accuracy.
In addition, for example, Patent Document 2 describes a method of computing a motion vector by means of resolving a part of the image on the target point side to sub-pixel accuracy and resolving the whole image on the corresponding point side to sub-pixel accuracy to search the corresponding point during the template matching for the correspondence search. As a result, the search can be performed at the sub-pixel level, allowing the calculation of the motion vectors with high accuracy.
Furthermore, for example, Patent Document 3 describes a correspondence search method. Described is a method in which stereo images that are captured at different timings are used for the corresponding point search to generate distance information and a two-dimensional motion vector at each timing, and based on which a three-dimensional motion vector is generated. With this method, the target point and the corresponding point are given at the pixel level.
In addition, for example, Non-patent Document 1 discloses a technique for high-accuracy correspondence search using Phase-Only Correlation (POC). In this technique, a corresponding point at the pixel level in the reference image is calculated which corresponds to the target point at the pixel level in the image of interest. Then, the amount of displacement from the corresponding point is estimated at the sub-pixel level. The corresponding point at the sub-pixel level is calculated while considering the amount of displacement into the calculated corresponding point.
However, although the method disclosed in the Patent Document 1 calculates the corresponding point at the sub-pixel level, this corresponding point is used as the target point for the correspondence search in a subsequent time-series image, during which the target point is defined at the pixel level rather than the sub-pixel level. This means that the method disclosed in the Patent Document 1 converts the corresponding point found at the sub-pixel level into the one at the pixel level to find the corresponding point in the subsequent image. Thus, the method disclosed in this Patent Document 1 does not compute the motion vectors in a consecutive manner for a plurality of time-series images.
In the method disclosed in the Patent Document 2, the sub-pixel versions of the images are created with a restricted predetermined resolution. Accordingly, the method disclosed in this Patent Document 2 has a problem of errors in estimation accuracy. In addition, creating the sub-pixel versions of the images causes a problem of increasing operation time.
Moreover, the method disclosed in the Patent Document 3 does not consider the case where the position of the corresponding point is at the sub-pixel level. Accordingly, in the method disclosed in this Patent Document 3, the correspondence search will be performed to a position at the pixel level located near the position at the sub-pixel level even when it is at a sub-pixel level position. This means that an offset will be produced from the correct position of the corresponding point. It is impossible to obtain an exact value when distance information, a two-dimensional motion vector and a three-dimensional motion vector are computed based on the corresponding point that has thus found. Even if a value that is closer to the exact one can be obtained by means of interpolating the resulting information about the corresponding point, it is after all the interpolated one and is far from a highly-accurate one.
Furthermore, the method disclosed in the Non-patent Document 1, computes the amount of displacement between the target point and the corresponding point at the sub-pixel level during the intermediate stage of the operation, during which the target point in the image of interest is converted for operation into the point at the sub-pixel level. However, the search is not performed with the target point defined at the sub-pixel level. Accordingly, as in the case of the method disclosed in the Patent Document 1, the correct position of the corresponding point is not computed, and highly-accurate matching is therefore not obtained.    Patent Document 1: Japanese Patent Application Laid-Open No. H5-236452    Patent Document 2: Japanese Patent Application Laid-Open No. 2007-257026    Patent Document 3: Japanese Patent Application Laid-Open No. 2001-84383    Non-patent Document 1: Kenji TAKITA, Mohammad Abdul MUQUIT, Takafumi AOKI, Tatsuo HIGUCHI, “A Sub-Pixel Correspondence Search Technique for Computer Vision Applications”, IEICE Transactions. Fundamentals, August 2004, E87-A, no. 8, pp. 1913-1923